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FOREWORD 


Results  of  measurements  and  conclusions  derived  from  them 
constitute  much  of  the  technical  information  produced  by 
NIST.  It  is  generally  agreed  that  the  usefulness  of 
measurement  results,  and  thus  much  of  the  information  that 
we  provide  as  an  institution,  is  to  a  large  extent  determined 
by  the  quality  of  the  statements  of  uncertainty  that 
accompany  them.  For  example,  only  if  quantitative  and 
thoroughly  documented  statements  of  uncertainty  accompany 
the  results  of  NIST  calibrations  can  the  users  of  our 
calibration  services  establish  their  level  of  traceability  to  the 
U.S.  standards  of  measurement  maintained  at  NIST. 

Although  the  vast  majority  of  NIST  measurement  results  are 
accompanied  by  quantitative  statements  of  uncertainty,  there 
has  never  been  a  uniform  approach  at  NIST  to  the 
expression  of  uncertainty.  The  use  of  a  single  approach 
within  the  Institute  rather  than  many  different  approaches 
would  ensure  the  consistency  of  our  outputs,  thereby 
simplifying  their  interpretation. 

To  address  this  issue,  in  July  1992  I  appointed  a  NIST  Ad 
Hoc  Committee  on  Uncertainty  Statements  and  charged  it 
with  recommending  to  me  a  NIST  policy  on  this  important 
topic.  The  members  of  the  Committee  were: 

D.  C.  Cranmer 

Materials  Science  and  Engineerijig  Laboratory 

K.  R.  Eberhardt 

Computing  and  Applied  Mathematics  Laboratory 

R.  M.  Judish 

Electronics  and  Electrical  Engineering  Laboratory 

R.  A.  Kamper 

Office  of  the  Director,  NIST/Boulder  Laboratories 

C.  E.  Kuyatt 
Physics  Laboratory 

J.  R.  Rosenblatt 

Computing  and  Applied  Mathematics  Laboratory 

J.  D.  Simmons 
Technology  Services 

L.  E.  Smith 

Office  of  the  Director,  NIST;  Chair 

D.  A.  Swyt 

Manufacturing  Engineering  Laboratory 

B.  N.  Taylor 
Physics  Laboratory 

R.  L.  Watters 

Chemical  Science  and  Technology  Laboratory 


This  action  was  motivated  in  part  by  the  emerging 
international  consensus  on  the  approach  to  expressing 
uncertainty  in  measurement  recommended  by  the 
International  Committee  for  Weights  and  Measures  (CIPM). 
The  movement  toward  the  international  adoption  of  the 
CIPM  approach  for  expressing  uncertainty  is  driven  to  a 
large  extent  by  the  global  economy  and  marketplace;  its 
worldwide  use  will  allow  measurements  performed  in 
different  countries  and  in  sectors  as  diverse  as  science, 
engineering,  commerce,  industry,  and  regulation  to  be  more 
easily  understood,  interpreted,  and  compared. 

At  my  request,  the  Ad  Hoc  Committee  carefully  reviewed 
the  needs  of  NIST  customers  regarding  statements  of 
uncertainty  and  the  compatibility  of  those  needs  with  the 
CIPM  approach.  It  concluded  that  the  CIPM  approach  could 
be  used  to  provide  quantitative  expressions  of  measurement 
uncertainty  that  would  satisfy  our  customers'  requirements. 
The  Ad  Hoc  Committee  then  recommended  to  me  a  specific 
policy  for  the  implementation  of  that  approach  at  NIST.  I 
enthusiastically  accepted  its  recommendation  and  the  policy 
has  been  incorporated  in  the  NIST  Administrative  Manual. 
(It  is  also  included  in  this  Technical  Note  as  Appendix  C.) 

To  assist  the  NIST  staff  in  putting  the  policy  into  practice, 
two  members  of  the  Ad  Hoc  Committee  prepared  this 
Technical  Note.  I  believe  that  it  provides  a  helpful  discussion 
of  the  CIPM  approach  and,  with  its  aid,  that  the  NIST  policy 
can  be  implemented  without  excessive  difficulty.  Further,  I 
believe  that  because  NIST  statements  of  uncertainty  resulting 
from  the  policy  will  be  uniform  among  themselves  and 
consistent  with  current  international  practice,  the  policy  will 
help  our  customers  increase  their  competitiveness  in  the 
national  and  international  marketplaces. 

January  1993 


John  W.  Lyons 
Director, 
National  Institute  of  Standards  and  Technology 


GUIDELINES  FOR  EVALUATING  AND  EXPRESSING  THE 
UNCERTAINTY  OF  NIST  MEASUREMENT  RESULTS 


1.  Introduction 

1.1  In  October  1992,  a  new  policy  on  expressing 
measurement  uncertainty  was  instituted  at  NIST.  This  policy 
is  set  forth  in  "Statements  of  Uncertainty  Associated  With 
Measurement  Results,"  Appendix  E,  NIST  Technical 
Communications  Program,  Subchapter  4.09  of  the 
Administrative  Manual  (reproduced  as  Appendix  C  of  these 
Guidelines). 

1.2  The  new  NIST  policy  is  based  on  the  approach  to 
expressing  uncertainty  in  measurement  recommended  by  the 
CIPM^  in  1981  [1]  and  the  elaboration  of  that  approach 
given  in  the  Guide  to  the  Expression  of  Uncertainty  in 
Measurement  (hereafter  called  the  Guide),  which  is  being 
prepared  by  individuals  nominated  by  the  BIPM,  lEC,  ISO, 
or  OIML  [2].'  The  CIPM  approach  is  founded  on 
Recommendation  INC-1  (1980)  of  the  Working  Group  on  the 
Statement  of  Uncertainties  [3].  This  group  was  convened  in 
1980  by  the  BIPM  as  a  consequence  of  a  1978  request  by 
the  CIPM  that  the  BIPM  study  the  question  of  reaching  an 
international  consensus  on  expressing  uncertainty  in 
measurement.  The  request  was  initiated  by  then  CIPM 
member  and  NBS  Director  E.  Ambler.  A  1986  request  by 
the  CIPM  to  ISO  asking  it  to  develop  a  broadly  applicable 
guidance  document  based  on  Recommendation  INC-1  (1980) 
led  to  the  development  of  the  Guide.  It  is  at  present  the  most 
complete  reference  on  the  general  application  of  the  CIPM 
approach  to  expressing  measurement  uncertainty,  and  its 
development  is  giving  further  impetus  to  the  worldwide 
adoption  of  that  approach. 

1 .3  Although  the  Guide  represents  the  current  international 
view  of  how  to  express  uncertainty  in  measurement  based  on 
the  CIPM  approach,  it  is  a  rather  lengthy  document.  We 
have  therefore  prepared  this  Technical  Note  with  the  goal  of 
succinctly  presenting,  in  the  context  of  the  new  NIST  policy, 


'CIPM:  International  Committee  for  Weights  and  Measures;  BIPM: 
International  Bureau  of  Weights  and  Measures;  lEC:  International 
Electrotechnical  Commission;  ISO:  International  Organization  for 
Standardization;  OIML:  International  Organization  of  Legal  Metrology. 


those  aspects  of  the  Guide  that  will  be  of  most  use  to  the 
NIST  staff  in  implementing  that  policy.  We  have  also 
included  some  suggestions  that  are  not  contained  in  the 
Guide  or  policy  but  which  we  believe  are  useful.  However, 
none  of  the  guidance  given  in  this  Technical  Note  is  to  be 
interpreted  as  NIST  policy  unless  it  is  directly  quoted  from 
the  policy  itself.  Such  cases  will  be  clearly  indicated  in  the 
text. 

1.4  The  guidance  given  in  this  Technical  Note  is  intended 
to  be  applicable  to  most,  if  not  all,  NIST  measurement 
results,  including  results  associated  with 

-  international  comparisons  of  measurement  standards, 

-  basic  research, 

-  applied  research  and  engineering, 

-  calibrating  client  measurement  standards, 

-  certifying  standard  reference  materials,  and 

-  generating  standard  reference  data. 

Since  the  Guide  itself  is  intended  to  be  applicable  to  similar 
kinds  of  measurement  results,  it  may  be  consulted  for 
additional  details.  Classic  expositions  of  the  statistical 
evaluation  of  measurement  processes  are  given  in  references 
[4-7]. 

2.  Classiflcation  of  Components  of  Uncertainty 

2.1  In  general,  the  result  of  a  measurement  is  only  an 
approximation  or  estimate  of  the  value  of  the  specific 
quantity  subject  to  measurement,  that  is,  the  measurand, 
and  thus  the  result  is  complete  only  when  accompanied  by  a 
quantitative  statement  of  its  uncertainty. 

2.2  The  uncertainty  of  the  result  of  a  measurement 
generally  consists  of  several  components  which,  in  the  CIPM 
approach,  may  be  grouped  into  two  categories  according  to 
the  method  used  to  estimate  their  numerical  values: 

A.  those  which  are  evaluated  by  statistical  methods, 

B.  those  which  are  evaluated  by  other  means. 


2.3  There  is  not  always  a  simple  correspondence  between 
the  classification  of  uncertainty  components  into  categories 
A  and  B  and  the  commonly  used  classification  of  uncertainty 
components  as  "random"  and  "systematic."  The  nature  of  an 
uncertainty  component  is  conditioned  by  the  use  made  of  the 
corresponding  quantity,  that  is,  on  how  that  quantity  appears 
in  the  mathematical  model  that  describes  the  measurement 
process.  When  the  corresponding  quantity  is  used  in  a 
different  way,  a  "random"  component  may  become  a 
"systematic"  component  and  vice  versa.  Thus  the  terms 
"random  uncertainty"  and  "systematic  uncertainty"  can  be 
misleading  when  generally  applied.  An  alternative 
nomenclature  that  might  be  used  is 

"component  of  uncertainty  arising  from  a  random  effect," 

"component  of  uncertainty  arising  from  a  systematic 
effect," 

where  a  random  effect  is  one  that  gives  rise  to  a  possible 
random  error  in  the  current  measurement  process  and  a 
systematic  effect  is  one  that  gives  rise  to  a  possible 
systematic  error  in  the  current  measurement  process.  In 
principle,  an  uncertainty  component  arising  from  a 
systematic  effect  may  in  some  cases  be  evaluated  by  method 
A  while  in  other  cases  by  method  B  (see  subsection  2.2),  as 
may  be  an  uncertainty  component  arising  from  a  random 
effect. 

NOTE  -  The  difference  between  error  and  uncertainty  should  always  be 
borne  in  mind.  For  example,  the  result  of  a  measurement  after 
correction  (see  subsection  5.2)  can  unknowably  be  very  close  to  the 
unknown  value  of  the  measurand,  and  thus  have  negligible  error,  even 
though  it  may  have  a  large  uncertainty  (see  the  Guide  [2]). 

2.4  Basic  to  the  CIPM  approach  is  representing  each 
component  of  uncertainty  that  contributes  to  the  uncertainty 
of  a  measurement  result  by  an  estimated  standard  deviation, 
termed  standard  uncertainty  with  suggested  symbol  u^ ,  and 
equal  to  the  positive  square  root  of  the  estimated  variance 


2.5  It  follows  from  subsections  2.2  and  2.4  that  an 
uncertainty  component  in  category  A  is  represented  by  a 
statistically  estimated  standard  deviation  5,,  equal  to  the 
positive  square  root  of  the  statistically  estimated  variance  s^ , 
and  the  associated  number  of  degrees  of  freedom  Vj.  For 
such  a  component  the  standard  uncertainty  is  «,  =  Sj . 


The  evaluation  of  uncertainty  by  the  statistical  analysis  of 
series  of  observations  is  termed  a  Type  A  evaluation  (of 
uncertainty). 

2.6  In  a  similar  manner,  an  uncertainty  component  in 
category  B  is  represented  by  a  quantity  Uj,  which  may  be 
considered  an  approximation  to  the  corresponding  standard 
deviation;  it  is  equal  to  the  positive  square  root  of  uj ,  which 
may  be  considered  an  approximation  to  the  corresponding 
variance  and  which  is  obtained  from  an  assumed  probability 
distribution  based  on  all  the  available  information  (see 
section  4).  Since  the  quantity  uj  is  treated  like  a  variance 
and  Uj  like  a  standard  deviation,  for  such  a  component  the 
standard  uncertainty  is  simply  m  . 

The  evaluation  of  uncertainty  by  means  other  than  the 
statistical  analysis  of  series  of  observations  is  termed  a  Type 
B  evaluation  (of  uncertainty). 

2.7  Correlations  between  components  (of  either  category) 
are  characterized  by  estimated  covariances  [see  Appendix  A, 
Eq.  (A-3)]  or  estimated  correlation  coefficients. 

3.  Type  A  Evaluation  of  Standard  Uncertainty 

A  Type  A  evaluation  of  standard  uncertainty  may  be  based 
on  any  valid  statistical  method  for  treating  data.  Examples 
are  calculating  the  standard  deviation  of  the  mean  of  a  series 
of  independent  observations  [see  Appendix  A,  Eq.  (A-5)]; 
using  the  method  of  least  squares  to  fit  a  curve  to  data  in 
order  to  estimate  the  parameters  of  the  curve  and  their 
standard  deviations;  and  carrying  out  an  analysis  of  variance 
(ANOVA)  in  order  to  identify  and  quantify  random  effects 
in  certain  kinds  of  measurements.  If  the  measurement 
situation  is  especially  complicated,  one  should  consider 
obtaining  the  guidance  of  a  statistician.  The  NIST  staff  can 
consult  and  collaborate  in  the  development  of  statistical 
experiment  designs,  analysis  of  data,  and  other  aspects  of  the 
evaluation  of  measurements  with  the  Statistical  Engineering 
Division,  Computing  and  Applied  Mathematics  Laboratory. 
Inasmuch  as  this  Technical  Note  does  not  attempt  to  give 
detailed  statistical  techniques  for  carrying  out  Type  A 
evaluations,  references  [4-7],  and  reference  [8]  in  which  a 
general  approach  to  quality  control  of  measurement  systems 
is  set  forth,  should  be  consulted  for  basic  principles  and 
additional  references. 


4.  Type  B  Evaluation  of  Standard  Uncertainty 

4.1  A  Type  B  evaluation  of  standard  uncertainty  is  usually 
based  on  scientific  judgment  using  all  the  relevant 
information  available,  which  may  include 

-  previous  measurement  data, 

-  experience  with,  or  general  knowledge  of,  the 
behavior  and  property  of  relevant  materials  and 
instruments, 

-  manufacturer's  specifications, 

-  data  provided  in  calibration  and  other  reports,  and 

-  uncertainties  assigned  to  reference  data  taken  from 
handbooks. 

Some  examples  of  Type  B  evaluations  are  given  in 
subsections  4.2  to  4.6. 

4.2  Convert  a  quoted  uncertainty  that  is  a  stated  multiple 
of  an  estimated  standard  deviation  to  a  standard  uncertainty 
by  dividing  the  quoted  uncertainty  by  the  multiplier. 

4.3  Convert  a  quoted  uncertainty  that  defines  a 
"confidence  interval"  having  a  stated  level  of  confidence  (see 
subsection  5.5),  such  as  95  or  99  percent,  to  a  standard 
uncertainty  by  treating  the  quoted  uncertainty  as  if  a  normal 
distribution  had  been  used  to  calculate  it  (unless  otherwise 
indicated)  and  dividing  it  by  the  appropriate  factor  for  such 
a  distribution.  These  factors  are  1 .960  and  2.576  for  the  two 
levels  of  confidence  given  (see  also  the  last  line  of  Table  B.  1 
of  Appendix  B). 

4.4  Model  the  quantity  in  question  by  a  normal  distribution 
and  estimate  lower  and  upper  limits  a_  and  a+  such  that  the 
best  estimated  value  of  the  quantity  is  (a+  +  a_)/2  (i.e., 
the  center  of  the  limits)  and  there  is  1  chance  out  of  2  (i.e., 
a  50  percent  probability)  that  the  value  of  the  quantity  lies  in 
the  interval  a_  to  a+.  Then  Uj  =  1.48  a,  where 
a  =  (a+  -  a_)/2  is  the  half-width  of  the  interval. 

4.5  Model  the  quantity  in  question  by  a  normal  distribution 
and  estimate  lower  and  upper  limits  a_  and  a+  such  that  the 
best  estimated  value  of  the  quantity  is  (a+  +  a_)/2  and 
there  is  about  a  2  out  of  3  chance  (i.e.,  a  67  percent 
probability)  that  the  value  of  the  quantity  lies  in  the  mterval 
a_  to  a+.  Then  Uj  =  a,  where  a  =  (a+  -  a_)/2. 

4.6  Estimate  lower  and  upper  limits  a_  and  a+  for  the 
value  of  the  quantity  in  question  such  that  the  probability  that 


the  value  lies  in  the  interval  a.  to  a+  is,  for  all  practical 
purposes,  100  percent.  Provided  that  there  is  no 
contradictory  information,  treat  the  quantity  as  if  it  is  equally 
probable  for  its  value  to  lie  anywhere  within  the  interval  a_ 
to  a+;  that  is,  model  it  by  a  uniform  or  rectangular 
probability  distribution.  The  best  estimate  of  the  value  of  the 
quantity  is  then  (a+  +  a.)/2  with  Uj  =  a/V3,  where 
a  =  (a+  -  a_)/2. 

If  the  distribution  used  to  model  the  quantity  is  triangular 
rather  than  rectangular,  then  «   =  a/\/6 . 

If  the  quantity  in  question  is  modeled  by  a  normal 
distribution  as  in  subsections  4.4  and  4.5,  there  are  no  fmite 
limits  that  will  contain  100  percent  of  its  possible  values. 
However,  plus  and  minus  3  standard  deviations  about  the 
mean  of  a  normal  distribution  corresponds  to  99.73  percent 
limits.  Thus,  if  the  limits  a_  and  a+  of  a  normally 
distributed  quantity  with  mean  (a_^.  +  a_)/2  are  considered 
to  contain  "almost  all"  of  the  possible  values  of  the  quantity, 
that  is,  approximately  99.73  percent  of  them,  then  My  =  a/3, 
where  a  =  {a+  -  a_)l2. 

The  rectangular  distribution  is  a  reasonable  default  model  in 
the  absence  of  any  other  mformation.  But  if  it  is  known  that 
values  of  the  quantity  in  question  near  the  center  of  the 
limits  are  more  likely  than  values  close  to  the  limits,  a 
triangular  or  a  normal  distribution  may  be  a  better  model. 

4.7  Because  the  reliability  of  evaluations  of  components  of 
uncertainty  depends  on  the  quality  of  the  information 
available,  it  is  recommended  that  all  parameters  upon  which 
the  measurand  depends  be  varied  to  the  fullest  extent 
practicable  so  that  the  evaluations  are  based  as  much  as 
possible  on  observed  data.  Whenever  feasible,  the  use  of 
empirical  models  of  the  measurement  process  founded  on 
long-term  quantitative  data,  and  the  use  of  check  standards 
and  control  charts  that  can  indicate  if  a  measurement  process 
is  under  statistical  control,  should  be  part  of  the  effort  to 
obtain  reliable  evaluations  of  components  of  uncertainty  [8] . 
Type  A  evaluations  of  uncertainty  based  on  limited  data  are 
not  necessarily  more  reliable  than  Type  B  evaluations. 


5.  Combined  Standard  Uncertainty 

5.1  The  combined  standard  uncertainty  of  a  measure- 
ment result,  suggested  symbol  u^,  is  taken  to  represent  the 
estimated  standard  deviation  of  the  result.  It  is  obtained  by 


combining  the  individual  standard  uncertainties  «,  (and 
covariances  as  appropriate),  whether  arising  from  a  Type  A 
evaluation  or  a  Type  B  evaluation,  using  the  usual  method 
for  combining  standard  deviations.  This  method,  which  is 
summarized  in  Appendix  A  [Eq.  (A-3)],  is  often  called  the 
law  of  propagation  of  uncertainty  and  in  common  parlance 
the  "root-sum-of-squares"  (square  root  of  the  sum-of-the- 
squares)  or  "RSS"  method  of  combining  uncertainty 
components  estimated  as  standard  deviations. 

NOTE  -  The  NIST  policy  also  allows  the  use  of  established  and 
documented  methods  equivalent  to  the  "RSS"  method,  such  as  the 
numerically  based  "bootstrap"  (see  Appendix  C). 

5.2  It  is  assumed  that  an  estimated  correction  (or 
correction  factor)  is  applied  to  compensate  for  each 
recognized  systematic  effect  that  significantly  influences  the 
measurement  result  and  that  every  effort  has  been  made  to 
identify  such  effects.  The  relevant  uncertainty  to  associate 
with  each  recognized  systematic  effect  is  then  the  standard 
uncertainty  of  the  applied  correction.  The  correction  may  be 
either  positive,  negative,  or  zero,  and  its  standard 
uncertainty  may  in  some  cases  be  obtained  from  a  Type  A 
evaluation  while  in  other  cases  by  a  Type  B  evaluation. 

NOTES 

1  The  uncertainty  of  an  estimated  correction  applied  to  a  measurement 
result  to  compensate  for  a  systematic  effect  is  not  the  systematic  error 
in  the  measurement  result  due  to  the  effect.  Rather,  it  is  a  measure  of 
the  uncertainty  of  the  result  due  to  incomplete  knowledge  of  the  value 
of  the  correction.  The  terms  "error"  and  "uncertainty"  should  not  be 
confused  (see  also  the  note  of  subsection  2.3). 

2  Although  it  is  strongly  recommended  that  corrections  be  applied  for 
all  recognized  significant  systematic  effects,  in  some  cases  it  may  not  be 
practical  because  of  limited  resources.  Nevertheless,  the  expression  of 
uncertainty  in  such  cases  should  conform  with  these  guidelines  to  the 
fullest  possible  extent  (see  the  Guide  [2]). 

5.3  The  combined  standard  uncertainty  u^  is  a  widely 
employed  measure  of  uncertainty.  The  NIST  policy  on 
expressing  uncertainty  states  that  (see  Appendix  C): 

Commonly,  u^  is  used  for  reporting  results  of 
determinations  of  fundamental  constants,  fundamental 
metrological  research,  and  international  comparisons 
of  realizations  of  SI  units. 

Expressing  the  uncertainty  of  NIST's  primary  cesium 
frequency  standard  as  an  estimated  standard  deviation  is  an 
example  of  the  use  of  u^  in  fundamental  metrological 
research.    It    should    also    be    noted    that    in    a    1986 


recommendation  [9],  the  CIPM  requested  that  what  is  now 
termed  combined  standard  uncertainty  u^  be  used  "by  all 
participants  in  giving  the  results  of  all  international 
comparisons  or  other  work  done  under  the  auspices  of  the 
CIPM  and  Comites  Consuhatifs." 

5.4  In  many  practical  measurement  situations,  the 
probability  distribution  characterized  by  the  measurement 
result  y  and  its  combined  standard  uncertainty  ujj)  is 
approximately  normal  (Gaussian).  When  this  is  the  case  and 
u^{y)  itself  has  negligible  uncertainty  (see  Appendix  B),  ujj) 
defines  an  interval  y  -  u^{y)  \o  y  +  u^(y)  about  the 
measurement  result  y  within  which  the  value  of  the 
measurand  Y  estimated  by  y  can  be  asserted  to  lie  with  a 
level  of  confidence  of  approximately  68  percent.  That  is,  it 
may  be  asserted  with  an  approximate  level  of  confidence  of 
68  percent  that  >'  -  u^iy)  <  Y  <  y  +  u^(y),  which  is 
commonly  written  as  Y  =  y  ±  u^(y). 

The  probability  distribution  characterized  by  the 
measurement  result  and  its  combined  standard  uncertainty  is 
approximately  normal  when  the  conditions  of  the  Central 
Limit  Theorem  are  met.  This  is  the  case,  often  encountered 
in  practice,  when  the  estimate  y  of  the  measurand  Y  is  not 
determined  directly  but  is  obtained  from  the  estimated  values 
of  a  significant  number  of  other  quantities  [see  Appendix  A, 
Eq.  (A-1)]  describable  by  well-behaved  probability 
distributions,  such  as  the  normal  and  rectangular 
distributions;  the  standard  uncertainties  of  the  estimates  of 
these  quantities  contribute  comparable  amounts  to  the 
combined  standard  uncertainty  uj^y)  of  the  measurement 
result  y;  and  the  linear  approximation  implied  by  Eq.  (A-3) 
in  Appendix  A  is  adequate. 

NOTE  -  If  u^iy)  has  non-negligible  uncertainty,  the  level  of  confidence 
will  differ  from  68  percent.  The  procedure  given  in  Appendix  B  has 
been  proposed  as  a  simple  expedient  for  approximating  the  level  of 
confidence  in  these  cases. 

5.5  The  term  "confidence  interval"  has  a  specific 
definition  in  statistics  and  is  only  applicable  to  intervals 
based  on  u^  when  certain  conditions  are  met,  including  that 
all  components  of  uncertainty  that  contribute  to  u^  be 
obtained  from  Type  A  evaluations.  Thus,  in  these  guidelines, 
an  interval  based  on  u^  is  viewed  as  encompassing  a  fraction 
p  of  the  probability  distribution  characterized  by  the 
measurement  result  and  its  combined  standard  uncertainty, 
and  p  is  the  coverage  probability  or  level  of  confidence  of 
the  interval. 


6.  Expanded  Uncertainty 

6.1  Although  the  combined  standard  uncertainty  u^  is  used 
to  express  the  uncertainty  of  many  NIST  measurement 
results,  for  some  commercial,  industrial,  and  regulatory 
applications  of  NIST  results  (e.g.,  when  health  and  safety 
are  concerned),  what  is  often  required  is  a  measure  of 
uncertainty  that  defines  an  interval  about  the  measurement 
result  y  within  which  the  value  of  the  measurand  Y  can  be 
confidently  asserted  to  lie.  The  measure  of  uncertainty 
intended  to  meet  this  requirement  is  termed  expanded 
uncertainty,  suggested  symbol  U,  and  is  obtained  by 
multiplying  u^-iy)  by  a  coverage  factor,  suggested  symbol  k. 
Thus  U  =  ku^iy)  and  it  can  be  confidently  asserted  that 
y-U<Y<y  +  U,  which  is  commonly  written  as 
Y  =  y  ±U. 

It  is  to  be  understood  that  subsection  5.5  also  applies  to  the 
interval  defined  by  expanded  uncertainty  U. 

6.2  In  general,  the  value  of  the  coverage  factor  k  is  chosen 
on  the  basis  of  the  desired  level  of  confidence  to  be 
associated  with  the  interval  defined  by  U  =  ku^.  Typically, 
kis'va.  the  range  2  to  3 .  When  the  normal  distribution  applies 
and  M(.  has  negligible  uncertainty  (see  subsection  5.4), 
U  =  2u^  (\.Q.,  k  =  T)  defines  an  interval  having  a  level  of 
confidence  of  approximately  95  percent,  and  U  =  3M(,  (i.e., 
k  =  3)  defines  an  interval  having  a  level  of  confidence 
greater  than  99  percent. 

NOTE  -  For  a  quantity  z  described  by  a  normal  distribution  with 
expectation  ^^  ^'^'^  standard  deviation  a,  the  interval  (i^  +  ka 
encompasses 68. 27,  90,  95.45, 99,  and  99.73  percentof  the  distribution 
for  A:  =  1,  ^  =  1.645,  k  =  2,  k  =  2.576,  and  k  =  3,  respectively  (see 
the  last  line  of  Table  B.  1  of  Appendix  B). 

6.3  Ideally,  one  would  like  to  be  able  to  choose  a  specific 
value  of  k  that  produces  an  interval  corresponding  to  a  well- 
defined  level  of  confidence  p,  such  as  95  or  99  percent; 
equivalently,  for  a  given  value  of  k,  one  would  like  to  be 
able  to  state  unequivocally  the  level  of  confidence  associated 
with  that  interval.  This  is  difficult  to  do  in  practice  because 
it  requires  knowing  in  considerable  detail  the  probability 
distribution  of  each  quantity  upon  which  the  measurand 
depends  and  combining  those  distributions  to  obtain  the 
distribution  of  the  measurand. 

NOTE  -  The  more  thorough  the  investigation  of  the  possible  existence 
of  non-trivial  systematic  effects  and  the  more  complete  the  data  upon 


which  the  estimates  of  the  corrections  for  such  effects  are  based,  the 
closer  one  can  get  to  this  ideal  (see  subsections  4.7  and  5.2). 

6.4  The  CIPM  approach  does  not  specify  how  the  relation 
between  ^  and  /?  is  to  be  established.  The  Guide  [2]  and 
Dietrich  [10]  give  an  approximate  solution  to  this  problem 
(see  Appendix  B);  it  is  possible  to  implement  others  which 
also  approximate  the  result  of  combining  the  probability 
distributions  assumed  for  each  quantity  upon  which  the 
measurand  depends,  for  example,  solutions  based  on 
numerical  methods. 

6.5  In  light  of  the  discussion  of  subsections  6.1-6.4,  and 
in  keeping  with  the  practice  adopted  by  other  national 
standards  laboratories  and  several  metrological  organizations, 
the  stated  NIST  policy  is  (see  Appendix  C): 

Use  expanded  uncertainty  U  to  report  the  results  of  all 
NIST  measurements  other  than  those  for  which  u^  has 
traditionally  been  employed.  To  be  consistent  with 
current  international  practice,  the  value  oik\o  be  used 
at  NIST  for  calculating  U  is,  by  convention,  k  =  2. 
Values  of  k  other  than  2  are  only  to  be  used  for 
specific  applications  dictated  by  established  and 
documented  requirements. 

An  example  of  the  use  of  a  value  of  k  other  than  2  is  taking 
k  equal  to  a  r-factor  obtained  from  the  ^-distribution  when  u^ 
has  low  degrees  of  freedom  in  order  to  meet  the  dictated 
requirement  of  providing  a  value  of  f7  =  ku^  that  defines  an 
interval  having  a  level  of  confidence  close  to  95  percent. 
(See  Appendix  B  for  a  discussion  of  how  a  value  of  k  that 
produces  such  a  value  of  U  might  be  approximated.) 

6.6  The  NIST  policy  provides  for  exceptions  as  follows 
(see  Appendix  C): 

It  is  understood  that  any  valid  statistical  method  that  is 
technically  justified  under  the  existing  circumstances 
may  be  used  to  determine  the  equivalent  of  m,  ,  Mj,  ,  or 
U.  Further,  it  is  recognized  that  international,  national, 
or  contractual  agreements  to  which  NIST  is  a  party 
may  occasionally  require  deviation  from  NIST  policy. 
In  both  cases,  the  report  of  uncertainty  must  document 
what  was  done  and  why. 

7.  Reporting  Uncertainty 

7 . 1     The  stated  NIST  policy  regarding  reporting  uncertainty 


is  (see  Appendix  C): 

Report  U  together  with  the  coverage  factor  k  used  to 
obtain  it,  or  report  u^. 

When  reporting  a  measurement  result  and  its 
uncertainty,  include  the  following  information  in  the 
report  itself  or  by  referring  to  a  published  document: 

-  A  list  of  all  components  of  standard  uncertainty, 
together  with  their  degrees  of  freedom  where 
appropriate,  and  the  resulting  value  of  u^.  The 
components  should  be  identified  according  to  the 
method  used  to  estimate  their  numerical  values: 

A.  those  which  are  evaluated  by  statistical 
methods, 

B.  those  which  are  evaluated  by  other  means. 

-  A  detailed  description  of  how  each  component  of 
standard  uncertainty  was  evaluated. 

-  A  description  of  how  k  was  chosen  when  k  is  not 
taken  equal  to  2. 

It  is  often  desirable  to  provide  a  probability 
interpretation,  such  as  a  level  of  confidence,  for  the 
interval  defined  by  U  ox  u^.  When  this  is  done,  the 
basis  for  such  a  statement  must  be  given. 

7.2  The  NIST  requirement  that  a  full  description  of  what 
was  done  be  given  is  in  keeping  with  the  generally  accepted 
view  that  when  reporting  a  measurement  result  and  its 
uncertainty,  it  is  preferable  to  err  on  the  side  of  providing 
too  much  information  rather  than  too  little.  However,  when 
such  details  are  provided  to  the  users  of  NIST  measurement 
results  by  referring  to  published  documents,  which  is  often 
the  case  when  such  results  are  given  in  calibration  and  test 
reports  and  certificates,  it  is  imperative  that  the  referenced 
documents  be  kept  up-to-date  so  that  they  are  consistent  with 
the  measurement  process  in  current  use. 

7.3  The  last  paragraph  of  the  NIST  policy  on  reporting 
uncertainty  (see  subsection  7.1  above)  refers  to  the 
desirability  of  providing  a  probability  interpretation,  such  as 
a  level  of  confidence,  for  the  interval  defined  by  U  or  u^. 
The  following  examples  show  how  this  might  be  done  when 
the  numerical  result  of  a  measurement  and  its  assigned 
uncertainty  is  reported,  assuming  that  the  published  detailed 
description  of  the  measurement  provides  a  sound  basis  for 


the  statements  made.  (In  each  of  the  three  cases,  the  quantity 
whose  value  is  being  reported  is  assumed  to  be  a  nominal 
100  g  standard  of  mass  m^.) 

m^  =  (100.021  47  +  0.000  70)  g,  where  the  + 
uncertainty  is  an  expanded  uncertainty  U  =  ku^,  with  U 
determined  from  a  combined  standard  uncertainty  (i.e., 
estimated  standard  deviation)  m^  =  0.35  mg  and  a 
coverage  factor  k  =  2.  Since  it  can  be  assumed  that  the 
possible  estimated  values  of  the  standard  are 
approximately  normally  distributed  with  approximate 
standard  deviation  u^ ,  the  unknown  value  of  the  standard 
can  be  asserted  to  lie  in  the  interval  defmed  by  U  with  a 
level  of  confidence  of  approximately  95  percent. 

m,  =  (100.021  47  +  0.000  79)  g,  where  the  ± 
uncertainty  is  an  expanded  uncertainty  U  =  ku^,  with  U 
determined  from  a  combined  standard  uncertainty  (i.e., 
estimated  standard  deviation)  u^  =  0.35  mg  and  a 
coverage  factor  k  =  2.26  based  on  the  f-distribution  for 
V  =  9  degrees  of  freedom,  and  defines  an  interval  within 
which  the  unknown  value  of  the  standard  can  be  asserted 
to  lie  with  a  level  of  confidence  of  approximately  95 
percent. 

wis  —  100.021  47  g  with  a  combined  standard  uncertainty 
(i.e.,  estimated  standard  deviation)  of  u^  =  0.35  mg. 
Since  it  can  be  assumed  that  the  possible  estimated  values 
of  the  standard  are  approximately  normally  distributed 
with  approximate  standard  deviation  u^,  the  unknown 
value  of  the  standard  can  be  asserted  to  lie  in  the  interval 
Wj  ±  «£  with  a  level  of  confidence  of  approximately  68 
percent. 

When  providing  such  probability  interpretations  of  the 
intervals  defined  by  U  and  u^,  subsection  5.5  should  be 
recalled.  In  this  regard,  the  interval  defined  by  f/  in  the 
second  example  might  be  a  conventional  confidence  interval 
(at  least  approximately)  if  all  the  components  of  uncertainty 
are  obtained  from  Type  A  evaluations. 

7.4  Some  users  of  NIST  measurement  results  may 
automatically  interpret  U  =  2u^  and  u^  as  quantities  that 
define  intervals  having  levels  of  confidence  corresponding  to 
those  of  a  normal  distribution,  namely,  95  percent  and  68 
percent,  respectively.  Thus,  when  reporting  either  U  =  2u^ 
or  W(,,  if  it  is  known  that  the  interval  which  U  =  2u^ot  u^ 
defines  has  a  level  of  confidence  that  differs  significantly 
from  95  percent  or  68  percent,  it  should  be  so  stated  as  an 


aid  to  the  users  of  the  measurement  result.  In  keeping  with 
the  NIST  policy  quoted  in  subsection  6.5,  when  the  measure 
of  uncertainty  is  expanded  uncertainty  U,  one  may  use  a 
value  of  k  that  does  lead  to  a  value  of  U  that  defines  an 
interval  having  a  level  of  confidence  of  95  percent  if  such  a 
value  of  U  is  necessary  for  a  specific  application  dictated  by 
an  established  and  documented  requirement. 

7.5  In  general,  it  is  not  possible  to  know  in  detail  all  of  the 
uses  to  which  a  particular  NIST  measurement  result  will  be 
put.  Thus,  it  is  usually  inappropriate  to  include  in  the 
uncertainty  reported  for  a  NIST  result  any  component  that 
arises  from  a  NIST  assessment  of  how  the  result  might  be 
employed;  the  quoted  uncertainty  should  normally  be  the 
actual  uncertainty  obtained  at  NIST. 

7.6  It  follows  from  subsection  7.5  that  for  standards  sent 
by  customers  to  NIST  for  calibration,  the  quoted  uncertainty 
should  not  normally  include  estimates  of  the  uncertainties 
that  may  be  introduced  by  the  return  of  the  standard  to  the 
customer's  laboratory  or  by  its  use  there  as  a  reference 
standard  for  other  measurements.  Such  uncertainties  are  due, 
for  example,  to  effects  arising  from  transportation  of  the 
standard  to  the  customer's  laboratory,  including  mechanical 
damage;  the  passage  of  time;  and  differences  between  the 
environmental  conditions  at  the  customer's  laboratory  and  at 
NIST.  A  caution  may  be  added  to  the  reported  uncertainty 
if  any  such  effects  are  likely  to  be  significant  and  an 
additional  uncertainty  for  them  may  be  estimated  and  quoted. 
If,  for  the  convenience  of  the  customer,  this  additional 
uncertainty  is  combined  with  the  uncertainty  obtained  at 
NIST,  a  clear  statement  should  be  included  explaining  that 
this  has  been  done. 

Such  considerations  are  also  relevant  to  the  uncertainties 
assigned  to  certified  devices  and  materials  sold  by  NIST. 
However,  well-justified,  normal  NIST  practices,  such  as 
including  a  component  of  uncertainty  to  account  for  the 
instability  of  the  device  or  material  when  it  is  known  to  be 
significant,  are  clearly  necessary  if  the  assigned  uncertainties 
are  to  be  meaningful. 
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Appendix  A 

Law  of  Propagation  of  Uncertainty 

A.l  In  many  cases  a  measurand  Y  is  not  measured  directly, 
but  is  determined  from  A^  other  quantities  X^,  X2,  .  .  .  ,  X^ 
through  a  functional  relation/: 


Y=f(X,,X2,  .  .  .  ,Xj,). 


(A-1) 


Included  among  the  quantities  X,  are  corrections  (or 
correction  factors)  as  described  in  subsection  5.2,  as  well  as 
quantities  that  take  into  account  other  sources  of  variability, 
such  as  different  observers,  instruments,  samples, 
laboratories,  and  times  at  which  observations  are  made  (e.g., 
different  days).  Thus  the  function  /of  Eq.  (A-1)  should 
express  not  simply  a  physical  law  but  a  measurement 
process,  and  in  particular,  it  should  contain  all  quantities  that 
can  contribute  a  significant  uncertainty  to  the  measurement 
result. 

A. 2  An  estimate  of  the  measurand  or  output  quantity  Y, 
denoted  by  y,  is  obtained  from  Eq.  (A-1)  using  input 
estimates  x^,  X2,  .  .  .  ,  %  for  the  values  of  the  A'^  input 
quantities  X,,  X2,  .  .  .  ,  X^y.  Thus  the  output  estimate  y, 
which  is  the  result  of  the  measurement,  is  given  by 


y  =  fix, ,  X 


2 ' 


,%). 


(A-2) 


A. 3  The  combined  standard  uncertainty  of  the  measurement 
result  y,  designated  by  u^(y)  and  taken  to  represent  the 
estimated  standard  deviation  of  the  result,  is  the  positive 
square  root  of  the  estimated  variance  ul(y)  obtained  from 


Uo(y)  =  E 


1=1 


dx- 

N-l     N 


u\x,) 


(A-3) 


2^^^^u(x,,xj). 
,=1  ;=,+!  ax.  dXj 


Equation  (A-3)  is  based  on  a  first-order  Taylor  series 
approximation  of  Y  =  f(X^,  X2,  .  .  .  ,  X^)  and  is 
conveniently  referred  to  as  the  law  of  propagation  of 
uncertainty.  The  partial  derivatives  df/dXj  (often  referred  to 
as  sensitivity  coefficients)  are  equal  to  df/dXj  evaluated  at 
X,  =  Xi ;  m(a:,)  is  the  standard  uncertainty  associated  with  the 
input  estimate  x, ;  and  u(Xj ,  Xj)  is  the  estimated  covariance 
associated  with  x,  and  X: . 

A. 4  As  an  example  of  a  Type  A  evaluation,  consider  an 
input  quantity  X,  whose  value  is  estimated  from  n 
independent  observations  X,  ^  of  X,  obtained  under  the  same 
conditions  of  measurement.  In  this  case  the  input  estimate  x, 
is  usually  the  sample  mean 


~  ^i   ~   —  Z^^i.k  ' 


(A-4) 


n  rr 


k=l 


and  the  standard  uncertainty  m(x,)  to  be  associated  with  x,  is 
the  estimated  standard  deviation  of  the  mean 


u(x,)  =  s{X.) 


— ! — V(x  .-x.f 

n(n-l)ti     '■'       ' 


1/2         (A-5) 


A.5  As  an  example  of  a  Type  B  evaluation,  consider  an 
input  quantity  X,  whose  value  is  estimated  from  an  assumed 
rectangular  probability  distribution  of  lower  limit  a_  and 
upper  limit  a+.  In  this  case  the  input  estimate  is  usually  the 
expectation  of  the  distribution 


Xj  =  (a,+  a_)/2. 


(A-6) 


and  the  standard  uncertainty  m(x,)  to  be  associated  with  Xj  is 
the  positive  square  root  of  the  variance  of  the  distribution 


u{x-)  =  a/\p  , 


where  a  =  (a+  -  a_)/2  (see  subsection  4.6). 


(A-7) 


NOTE  -  When  x,  is  obtained  from  an  assumed  distribution,  the 
associated  variance  is  appropriately  written  as  m^(X,)  and  the  associated 
standard  uncertainty  as  u(Xi),  but  for  simplicity,  m^(x,)  and  m(a:,)  are 
used.  Similar  considerations  apply  to  the  symbols  uliy)  and  u^{y). 


Appendix  B 

Coverage  Factors 

B.l  This  appendix  summarizes  a  conventional  procedure, 
given  by  the  Guide  [2]  and  Dietrich  [10],  intended  for  use  in 
calculating  a  coverage  factor  k  when  the  conditions  of  the 
Central  Limit  Theorem  are  met  (see  subsection  5.4)  and  (1) 
a  value  other  than  ^  =  2  is  required  for  a  specific 
application  dictated  by  an  established  and  documented 
requirement;  and  (2)  that  value  of  k  must  provide  an  interval 
having  a  level  of  confidence  close  to  a  specified  value.  More 
specifically,  it  is  intended  to  yield  a  coverage  factor  kp  that 
produces  an  expanded  uncertainty  Up  -  kpU^(y)  that  defines 
an  interval  y-Up<Y<y  +  Up,  which  is  commonly 
written  as  Y  =  y  ±  Up,  having  an  approximate  level  of 
confidence  p. 

The  four-step  procedure  is  included  in  these  guidelines 
because  it  is  expected  to  find  broad  acceptance 
internationally,  due  in  part  to  its  computational  convenience, 
in  much  the  same  way  that  k  =  2  has  become  the 
conventional  coverage  factor.  However,  although  the 
procedure  is  based  on  a  proven  approximation,  it  should  not 
be  interpreted  as  being  rigourous  because  the  approximation 
is  extrapolated  to  situations  where  its  applicability  has  yet  to 
be  fully  investigated. 

B.2  To  estimate  the  value  of  such  a  coverage  factor 
requires  taking  into  account  the  uncertainty  of  u^(y),  that  is, 
how  well  u^iy)  estimates  the  standard  deviation  associated 
with  the  measurement  result.  For  an  estimate  of  the  standard 
deviation  of  a  normal  distribution,  the  degrees  of  freedom  of 
the  estimate,  which  depends  on  the  size  of  the  sample  on 
which  the  estimate  is  based,  is  a  measure  of  its  uncertainty. 
For  a  combined  standard  uncertainty  u^iy),  the  "effective 
degrees  of  freedom"  p^ff  of  u^(y),  which  is  approximated  by 
appropriately  combining  the  degrees  of  freedom  of  its 
components,  is  a  measure  of  its  uncertainty.  Hence  v^f^  is  a 
key  factor  in  determining  kp .  For  example,  if  v^^f  is  less  than 
about  1 1 ,  simply  assuming  that  the  uncertainty  of  u^(  y)  is 
negligible  and  taking  k  =  2  may  be  inadequate  if  an 
expanded  uncertainty  U  =  ku^iy)  that  defines  an  interval 


having  a  level  of  confidence  close  to  95  percent  is  required 
for  a  specific  application.  More  specifically,  according  to 
Table  B.l  (to  be  discussed  below),  if  v^^f  =  8,  kg^  =  2.3 
rather  than  2.0.  In  this  case,  and  in  other  similar  cases 
where  i/gfr  of  u^(.y)  is  comparatively  small  and  an  interval 
having  a  level  of  confidence  close  to  a  specified  level  is 
required,  it  is  unlikely  that  the  uncertainty  of  u^(y)  would  be 
considered  negligible.  Instead,  the  small  value  of  j'^ff,  and 
thus  the  uncertainty  of  uj^y),  would  probably  be  taken  into 
account  when  determining  k  . 

B.3  The  four-step  procedure  for  calculating  kp  is  as 
follows: 

1)  Obtain  y  and  u^(y)  as  indicated  in  Appendix  A. 

2)  Estimate  the  effective  degrees  of  freedom  j'^ff  of  u^(y) 
from  the  Welch-Satterthwaite  formula 


Uc(y) 


"eff 


N        ^     4/      \ 


(B-l) 


where  c,  =  df/dx^,  all  of  the  u(Xj)  are  mutually  statistically 
independent,  Vj  is  the  degrees  of  freedom  of  m(x,),  and 


"eff    ^    E"/ 
1  =  1 


(B-2) 


The  degrees  of  freedom  of  a  standard  uncertainty  u{Xj) 
obtained  from  a  Type  A  evaluation  is  determined  by 
appropriate  statistical  methods  [7].  In  the  common  case 
discussedinsubsection  A.4  wherej:,  —  X,andM(jc,)  =  s(X^), 
the  degrees  of  freedom  of  u(Xj)  is  i/,  =  «  -  1.  If  m 
parameters  are  estimated  by  fitting  a  curve  to  n  data  points 
by  the  method  of  least  squares,  the  degrees  of  freedom  of 
the  standard  uncertainty  of  each  paramter  is  n  -  m. 

The  degrees  of  freedom  to  associate  with  a  standard 
uncertainty  u(Xj)  obtained  from  a  Type  B  evaluation  is  more 
problematic.  However,  it  is  common  practice  to  carry  out 
such  evaluations  in  a  manner  that  ensures  that  an 
underestimation  is  avoided.  For  example,  when  lower  and 
upper  limits  a_  and  a+  are  set  as  in  the  case  discussed  in 
subsection  A. 5,  they  are  usually  chosen  in  such  a  way  that 
the  probability  of  the  quantity  in  question  lying  outside  these 
limits  is  in  fact  extemely  small.  Under  the  assumption  that 


this  practice  is  followed,  the  degrees  of  freedom  of  u{Xj)  confidence  p  from  a  table  of  values  of  tpiv)  from  the 

may  be  taken  to  be  v^^  oo.  ^-distribution,  such  as  Table  B.l  of  this  Appendix.  If  v^f^  is 

not  an  integer,   which  will  usually  be  the  case,   either 

NOTE  -  See  the  Guide  [21  for  a  possible  way  to  estimate  v,  when  this  ■   ^         ,.          .           ,            »»i.           ^i 

i^wic    occ  uisp  ^uiuc  I  1         i>             3                 ,  mterpolate  or  truncate  p^ff  to  the  next  lower  mteger. 

assumption  is  not  justified. 

s    r       ,             •    J   ,      1     ^  4)  Take  k„  =  tJv^ff)  and  calculate  U„  =  k„uJy). 

3)  Obtain  the  /-factor  Uv^f^)  for  the  required  level  of  ^            p       p^  eff/                         p        p  c\yf 
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Table  B.l  —  Value  of  tp{v)  from  the  ^-distribution  for  degrees  of  freedom  v  that  defines  an  interval  -t  {v)  to 
+  tpiy)  that  encompasses  the  fraction  p  of  the  distribution 


Degrees  of 
frppHnm 

Fraction  p  in  percent 

V 

68. 27^^' 

90 

95 

95. 45^^' 

99 

99.73^"' 

1 

1.84 

6.31 

12.71 

13.97 

63.66 

235.8 

2 

1.32 

2.92 

4.30 

4.53 

9.92 

19.21 

3 

1.20 

2.35 

3.18 

3.31 

5.84 

9.22 

4 

1.14 

2.13 

2.78 

2.87 

4.60 

6.62 

5 

1.11 

2.02 

2.57 

2.65 

4.03 

5.51 

6 

1.09 

1.94 

2.45 

2.52 

3.71 

4.90 

7 

1.08 

1.89 

2.36 

2.43 

3.50 

4.53 

8 

1.07 

1.86 

2.31 

2.37 

3.36 

4.28 

9 

1.06 

1.83 

2.26 

2.32 

3.25 

4.09 

10 

1.05 

1.81 

2.23 

2.28 

3.17 

3.96 

11 

1.05 

1.80 

2.20 

2.25 

3.11 

3.85 

12 

1.04 

1.78 

2.18 

2.23 

3.05 

3.76 

13 

1.04 

1.77 

2.16 

2.21 

3.01 

3.69 

14 

1.04 

1.76 

2.14 

2.20 

2.98 

3.64 

15 

1.03 

1.75 

2.13 

2.18 

2.95 

3.59 

16 

1.03 

1.75 

2.12 

2.17 

2.92 

3.54 

17 

1.03 

1.74 

2.11 

2.16 

2.90 

3.51 

18 

1.03 

1.73 

2.10 

2.15 

2.88 

3.48 

19 

1.03 

1.73 

2.09 

2.14 

2.86 

3.45 

20 

1.03 

1.72 

2.09 

2.13 

2.85 

3.42 

25 

1.02 

1.71 

2.06 

2.11 

2.79 

3.33 

30 

1.02 

1.70 

2.04 

2.09 

2.75 

3.27 

35 

1.01 

1.70 

2.03 

2.07 

2.72 

3.23 

40 

1.01 

1.68 

2.02 

2.06 

2.70 

3.20 

45 

1.01 

1.68 

2.01 

2.06 

2.69 

3.18 

50 

1.01 

1.68 

2.01 

2.05 

2.68 

3.16 

100 

1.005 

1.660 

1.984 

2.025 

2.626 

3.077 

oo 

1.000 

1.645 

1.960 

2.000 

2.576 

3.000 

^^^For  a  quantity  z  described  by  a  normal 

distribution  with  expectation  \i^  and  standard  deviation  a,  the 

interval  \i.^  +  fca  encompasses  p  =  68.2' 

7,  95.45,  and  99.73  percent  of  the  distribution  ioxk  =  1,2, 

and  3,  respectively. 
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APPENDIX  E 

STATEMENTS  OF  UNCERTAINTY  ASSOCIATED  WITH 
MEASUREMENT  RESULTS 


A  measurement  result  is  complete  only  when  accompanied 
by  a  quantitative  statement  of  its  uncertainty.  This  policy 
requires  that  NIST  measurement  results  be  accompanied  by 
such  statements  and  that  a  uniform  approach  to  expressing 
measurement  uncertainty  be  followed. 

1.   Background 

Since  the  early  1980s,  an  international  consensus  has  been 
developing  on  a  uniform  approach  to  the  expression  of 
uncertainty  in  measurement.  Many  of  NIST's  sister  national 
standards  laboratories  as  well  as  a  number  of  important 
metrological  organizations,  including  the  Western  European 
Calibration  Cooperation  (WECC)  and  EUROMET,  have 
adopted  the  approach  recommended  by  the  International 
Committee  for  Weights  and  Measures  (CIPM)  in  1981  [1] 
and  reaffirmed  by  the  CIPM  in  1986  [2]. 

Equally  important,  the  CIPM  approach  has  come  into  use  in 
a  significant  number  of  areas  at  NIST  and  is  also  becoming 
accepted  in  U.S.  industry.  For  example,  the  National 
Conference  of  Standards  Laboratories  (NCSL)  is  using  it  to 
develop  a  Recommended  Practice  on  measurement 
uncertainty  for  NCSL  member  laboratories. 

The  CIPM  approach  is  based  on  Recommendation  INC-1 
(1980)  of  the  Working  Group  on  the  Statement  of 
Uncertainties  [3].  This  group  was  convened  in  1980  by  the 
International  Bureau  of  Weights  and  Measures  (BIPM)  in 
response  to  a  request  by  the  CIPM.  More  recently,  at  the 
request  of  the  CIPM,  a  joint  BIPM/IEC/ISO/OIML  working 
group  developed  a  comprehensive  reference  document  on  the 
general  application  of  the  CIPM  approach  titled  Guide  to  the 
Expression  of  Uncertainty  in  Measurement  [4]  (lEC: 
International     Electrotechnical     Commission;     ISO: 


International  Organization  for  Standardization;  OIML: 
International  Organization  of  Legal  Metrology).  The 
development  of  the  Guide  is  providing  further  impetus  to  the 
worldwide  adoption  of  the  CIPM  approach. 

2.   Policy 

All  NIST  measurement  results  are  to  be  accompanied  by 
quantitative  statements  of  uncertainty.  To  ensure  that  such 
statements  are  consistent  with  each  other  and  with  present 
international  practice,  this  NIST  policy  adopts  in  substance 
the  approach  to  expressing  measurement  uncertainty 
recommended  by  the  International  Committee  for  Weights 
and  Measures  (CIPM).  The  CIPM  approach  as  adapted  for 
use  by  NIST  is: 

1)  Staruiard  Uncertainty:  Represent  each  component  of 
uncertainty  that  contributes  to  the  uncertainty  of  the 
measurement  result  by  an  estimated  standard  deviation  «,, 
termed  standard  uncertainty,  equal  to  the  positive 
square  root  of  the  estimated  variance  m^. 

2)  Combined  Standard  Uncertainty:  Determine  the 
combined  standard  uncertainty  u^  of  the  measurement 
result,  taken  to  represent  the  estimated  standard  deviation 
of  the  result,  by  combining  the  individual  standard 
uncertainties  m,  (and  covariances  as  appropriate)  using  the 
usual  "root-sum-of-squares"  method,  or  equivalent 
established  and  documented  methods. 

Commonly,  u^  is  used  for  reporting  results  of 
determinations  of  fundamental  constants,  fundamental 
metrological  research,  and  international  comparisons  of 
realizations  of  SI  units. 
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3)  Expanded  Uncertainty:  Determine  an  expanded 
uncertainty  U  by  multiplying  u^  by  a  coverage  factor  k: 
U  =  ku^.  The  purpose  of  V  is  to  provide  an  interval 
y  -  Uioy  +  U about  the  result  y  within  which  the  value 
of  y,  the  specific  quantity  subject  to  measurement  and 
estimated  by  y,  can  be  asserted  to  lie  with  a  high  level  of 
confidence.  Thus  one  can  confidently  assert  that 
y-U<Y<y  +  U,  which  is  commonly  written  as 
Y  =  y  ±U. 


Additional  guidance  on  the  use  of  the  CIPM  approach  at 
NIST  may  be  found  in  Guidelines  for  Evaluating  and 
Expressing  the  Uncertainty  of  NIST  Measurement  Results  [5]. 
A  more  detailed  discussion  of  the  CIPM  approach  is  given 
in  the  Guide  to  the  Expression  of  Uncertainty  in 
Measurement  [4].  Classic  expositions  of  the  statistical 
evaluation  of  measurement  processes  are  given  in  references 
[6-8]. 


Use  expanded  uncertainty  U  to  report  the  results  of  all        3.    Responsibilities 


NIST  measurements  other  than  those  for  which  u^  has 
traditionally  been  employed.  To  be  consistent  with  current 
international  practice,  the  value  of  k  to  be  used  at  NIST 
for  calculating  U  is,  by  convention,  k  -  2.  Values  of  k 
other  than  2  are  only  to  be  used  for  specific  applications 
dictated  by  established  and  documented  requirements. 

4)  Reporting    Uncertainty:    Report    U  together   with   the 
coverage  factor  k  used  to  obtain  it,  or  report  u^ . 

When  reporting  a  measurement  result  and  its  uncertainty, 
include  the  following  information  in  the  report  itself  or  by 
referring  to  a  published  document: 

-  A  list  of  all  components  of  standard  uncertainty, 
together  with  their  degrees  of  freedom  where 
appropriate,  and  the  resulting  value  of  u^.  The 
components  should  be  identified  according  to  the 
method  used  to  estimate  their  numerical  values: 

A.  those    which    are    evaluated    by    statistical 
methods, 

B.  those  which  are  evaluated  by  other  means. 

-  A  detailed  description  of  how  each  component  of 
standard  uncertainty  was  evaluated. 

-  A  description  of  how  k  was  chosen  when  k  is  not 
taken  equal  to  2. 

It  is  often  desirable  to  provide  a  probability  interpretation, 
such  as  a  level  of  confidence,  for  the  interval  defined  by 
C/  or  Mg.  When  this  is  done,  the  basis  for  such  a  statement 
must  be  given. 


a.  Operating  Unit  Directors  are  responsible  for  compliance 
with  this  policy. 

b.  The  Statistical  Engineering  Division,  Computing  and 
Applied  Mathematics  Laboratory,  is  responsible  for 
providing  technical  advice  on  statistical  methods  for 
evaluating  and  expressing  the  uncertainty  of  NIST 
measurement  results. 

c.  NIST  Editorial  Review  Boards  are  responsible  for 
ensuring  that  statements  of  measurement  uncertainty  are 
included  in  NIST  publications  and  other  technical  outputs 
under  their  jurisdiction  which  report  measurement  results 
and  that  such  statements  are  in  conformity  with  this  policy. 

d.  The  Calibrations  Advisory  Group  is  responsible  for 
ensuring  that  calibration  and  test  reports  and  other  technical 
outputs  under  its  jurisdiction  are  in  compliance  with  this 
policy. 

e.  The  Standard  Reference  Materials  and  Standard  Reference 
Data  programs  are  responsible  for  ensuring  that  technical 
outputs  under  their  jurisdiction  are  in  compliance  with  this 
policy. 

f.  Authors,  as  part  of  the  process  of  preparing  manuscripts 
and  other  technical  outputs,  are  responsible  for  formulating 
measurement  uncertainty  statements  consistent  with  this 
policy.  These  statements  must  be  present  in  drafts  submitted 
for  NIST  review  and  approval. 
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4.  Exceptions 

It  is  understood  that  any  valid  statistical  method  that  is 
technically  justified  under  the  existing  circumstances  may  be 
used  to  determine  the  equivalent  of  m,,  u^,  or  U.  Further,  it 
is  recognized  that  international,  national,  or  contractual 
agreements  to  which  NIST  is  a  party  may  occasionally 
require  deviation  from  this  policy.  In  both  cases,  the  report 
of  uncertainty  must  document  what  was  done  and  why. 
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Technical  Publications 

Periodical 

Journal  of  Research  of  the  National  Institute  of  Standards  and  Technology- Reports  NIST 
research  and  development  in  those  disciplines  of  the  physical  and  engineering  sciences  in 
which  the  Institute  is  active.  These  include  physics,  chemistry,  engineering,  mathematics,  and 
computer  sciences.  Papers  cover  a  broad  range  of  subjects,  with  major  emphasis  on 
measurement  methodology  and  the  basic  technology  underlying  standardization.  Also  included 
from  time  to  time  are  survey  articles  on  topics  closely  related  to  the  Institute's  technical  and 
scientific  programs.  Issued  six  times  a  year. 

Nonperiodicals 

Monographs— Major  contributions  to  the  technical  literature  on  various  subjects  related  to  the 
Institute's  scientific  and  technical  activities. 

Handbooks -Recommended  codes  of  engineering  and  industrial  practice  (including  safety 
codes)  developed  in  cooperation  with  interested  industries,  professional  organizations,  and 
regulatory  bodies. 

Special  Publications -Include  proceedings  of  conferences  sponsored  by  NIST,  NIST  annual 
reports,  and  other  special  publications  appropriate  to  this  grouping  such  as  wall  charts,  pocket 
cards,  and  bibliographies. 

Applied  Mathematics  Series— Mathematical  tables,  manuals,  and  studies  of  special  interest  to 
physicists,  engineers,  chemists,  biologists,  mathematicians,  computer  programmers,  and  others 
engaged  in  scientific  and  technical  work. 

National  Standard  Reference  Data  Series  -  Provides  quantitative  data  on  the  physical  and 
chemical  properties  of  materials,  compiled  from  the  world's  literature  and  critically  evaluated. 
Developed  under  a  worldwide  program  coordinated  by  NIST  under  the  authority  of  the  National 
Standard  Data  Act  (Public  Law  90-396).  NOTE:  The  Journal  of  Physical  and  Chemical 
Reference  Data  (JPCRD)  is  published  bi-monthly  for  NIST  by  the  American  Chemical  Society 
(ACS)  and  the  American  Institute  of  Physics  (AlP).  Subscriptions,  reprints,  and  supplements  are 
available  from  ACS,  1155  Sixteenth  St.,  NW.,  Washington,  DC  20056. 
Building  Science  Series— Disseminates  technical  information  developed  at  the  Institute  on 
building  materials,  components,  systems,  and  whole  structures.  The  series  presents  research 
results,  test  methods,  and  performance  criteria  related  to  the  structural  and  environmental 
functions  and  the  durability  and  safety  characteristics  of  building  elements  and  systems. 
Technical  Notes— Studies  or  reports  which  are  complete  in  themselves  but  restrictive  in  their 
treatment  of  a  subject.  Analogous  to  monographs  but  not  so  comprehensive  in  scope  or 
definitive  in  treatment  of  the  subject  area.  Often  serve  as  a  vehicle  for  final  reports  of  work 
performed  at  NIST  under  the  sponsorship  of  other  government  agencies. 
Voluntary  Product  Standards -Developed  under  procedures  published  by  the  Department  of 
Commerce  in  Part  10,  Title  15,  of  the  Code  of  Federal  Regulations.  The  standards  establish 
nationally  recognized  requirements  for  products,  and  provide  all  concerned  interests  with  a 
basis  for  common  understanding  of  the  characteristics  of  the  products.  NIST  administers  this 
program  in  support  of  the  efforts  of  private-sector  standardizing  organizations. 
Consumer  Information  Series -Practical  information,  based  on  NIST  research  and  experience, 
covering  areas  of  interest  to  the  consumer.  Easily  understandable  language  and  illustrations 
provide  useful  background  knowledge  for  shopping  in  today's  technological  marketplace. 
Order  the  above  NIST  publications  from:  Superintendent  of  Documents,  Government  Printing 
Office,  Washington,  DC  20402. 

Order  the  following  NIST  publications- FIPS  and  NISTIRs-from  the  National  Technical 
Information  Service,  Springfield,  VA  22161. 

Federal  Information  Processing  Standards  Publications  (FIPS  PUB) -Publications  in  this  series 
collectively  constitute  the  Federal  Information  Processing  Standards  Register.  The  Register 
serves  as  the  official  source  of  information  in  the  Federal  Government  regarding  standards 
issued  by  NIST  pursuant  to  the  Federal  Property  and  Administrative  Services  Act  of  1949  as 
amended.  Public  Law  89-306  (79  Stat.  1127),  and  as  implemented  by  Executive  Order  11717 
(38  FR  12315,  dated  May  11,  1973)  and  Part  6  of  Title  15  CFR  (Code  of  Federal  Regulations). 
NIST  Interagency  Reports  (NISTIR)-A  special  series  of  interim  or  final  reports  on  work 
performed  by  NIST  for  outside  sponsors  (both  government  and  non-government).  In  general, 
initial  distribution  is  handled  by  the  sponsor;  public  distribution  is  by  the  National  Technical 
Information  Sen/ice,  Springfield,  VA  22161,  in  paper  copy  or  microfiche  form. 
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